A Hierarchical Model for Continuous Gesture Recognition Using Kinect

Human gesture recognition is an area, which has been studied thoroughly in recent years, and close to 100% recognition rates in restricted environments have been achieved, often either with single separated gestures in the input stream, or with computationally intensive systems. The results are unfortunately not as striking, when it comes to a continuous stream of gestures. In this paper we introduce a hierarchical system for gesture recognition for use in a gaming setting, with a continuous stream of data. Layer 1 is based on Nearest Neighbor Search and layer 2 uses Hidden Markov Models. The system uses features that are computed from Microsoft Kinect skeletons. We propose a new set of features, the relative angles of the limbs from Kinect’s axes to use in NNS. The new features show a 10 percent point increase in precision when compared with features from previously published results. We also propose a way of attributing recognised gestures with a force attribute, for use in gaming. The recognition rate in layer 1 is 68.2%, with an even higher rate for simple gestures. Layer 2 reduces the noise and has a average recognition rate of 85.1%. When some simple constraints are added we reach a precision of 90.5% with a recall of 91.4%.